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CLAIMS 

What is claimed is: 3 

1. A method for evaluating accuracy ,of an automatic 
location identification (ALI) system configured to locate a: 
wireless communication device. originating a call through a . 
wireless communication network, said method comprising: 

identifying a validation region in which a service area of 
a public safety answering point (PSAP) and a radio frequency 
(RF) -coverage area of said wireless communication network 
overlap; ^ .-»?." 

selecting a test scenario that represents a calling 
environment in which said call may occur;- 

choosing a test point in said validation region exhibiting 
said calling environment represented by said test scenario; 

determining a ground truth. of said test point; 

performing a test call from said. test point, said test call 
simulating said call; 

obtaining a reported location of said test point . from said 

ALI system in response to said performing operation; and 

. . i ' 

comparing said reported ■location -'• with said ground truth to 

determine said accuracy of said ALI system in response to 

predetermined accuracy parameters . 

2. A method as claimed in claim 1 wherein said identifying 
operation comprises: 

obtaining a first map of said service area; 
acquiring a second map of said RF coverage area; 
selecting unit areas that are common to each of said first 
and second maps; and 
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compiling said unit areas to form a validation region map 
identifying said validation region. 

3. A method as claimed in claim 2 wherein said acquiring 
operation comprises: ■ 

defining a wireless- service area of said wireless 
communication network approximately corresponding to said 
service area of said PSAP; 

: locating RF coverage holes within: said wireless service 
area, .said RF coverage holes; being regions of said wireless 
service area exhibiting a signal strength >of RF communication 
signals that is less. than a predetermined design threshold; and 

dissociating said RF coverage holes from said wireless 
service area to produce said second map of said RF coverage 
area ' -• . ■ ^ ; . ;• ■ 

4 . ,A method as claimed in claim 3 wherein , said identifying 
operation further comprises 

accessing a PSAP service area database to obtain said first 
map ; • ■ . ., 

accessing a service area" database of said wireless 
.communication network to define said wireless service area; 

•accessing an RF coverage area database that identifies said 
RF coverage holes within said wireless service area; and 

querying each of said PSAP service area database, said 
service area database, and said RF coverage area database to 
compile said unit areas that are common to each of said first 
and .second maps, said unit areas exhibiting RF signal strength 
of said RF communication signals that is greater than said 
predetermined design threshold. 
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5. A method as claimed in claim 1 further comprising: 
obtaining a validation region map of said validation 

region; 

populating said validation region map with geographic and 
land use features of said validation regions- 
partitioning said validation -region map into a plurality of 
sub-regions; and v* 

correlating each of said sub-regions with one of a 
plurality of test scenarios,' said selected test scenario being 
one of said .plurality of - test .scenarios, . and each of said test -■■ 
scenarios representing a unique calling environment in said 
validation region. ^ 

6. A method as claimed in claim 5 wherein said choosing 
operation comprises initiating a random selection process to : 
choose said test point from within said sub-regions correlated 
with said selected test scenario. * 

7. A method as claimed in. claim 5 wherein said call is an 
emergency call and said choosing operation comprises 
distinguishing said test point in response , to a f requency of 

•wireless calls previously originated within a boundary of one 
of said sub-regions correlated with said, selected test scenario 
relative to said frequency of said wireless calls previously 
originated within others of said sub-regions correlated with 
said . selected test scenario. 

8. A method as claimed in claim 5 wherein said call is an 
emergency call and said choosing operation comprises 
distinguishing said test point in response to a frequency of 
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wireless calls previously conducted within a boundary of one of 
said sub-regions correlated with said selected test scenario 
relative to said frequency of said wireless calls previously 
conducted within others of said sub-regions correlated with 
said selected test scenario. 

9 '. A method as claimed in claim 1 wherein said choosing 
operation comprises ascertaining, that said test point is a 
^viable test point for performing: said test call:, ■ said viable 
.test point being accessible and exhibiting said calling 
.environment represented by said test ^scenario . 

10. A method as. claimed. in claim 9 further comprising 
conducting a ground survey of said test point to ascertain that 
said: test point is said viable test point. r 

11. A method as claimed in claim .1 wherein: 
said method further comprises: 

obtaining a validation region map of said validation 
region; 

establishing an RF signal propagation model of- said 
wireless communication network in said validation 
region' map, said model representing propagation of RF 
communication signals; through said validation region; 
activating said ALI system within said simulated 
environment; and 
; locating said test point in said validation region map, 
said ground truth being determined from said validation 
region map; and 
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said performing operation comprises simulating said test 
call within said simulated environment to obtain said reported 
, location of said test point'. - • 

.12. A method as claimed in claim .1 wherein said test point 
is a first test point, said determining operation determines 
said- ground truth of said first ;test point at a first instant 
in time, said 'Obtaining operation obtains said, reported 
^location of said first test point at said first instant in 
time, and said method further comprises: 

continuing .said test call at a second test point; 

determining a second ground truth of said second test 
point; ; a . 

n obtaining a second reported location of said second test 
point from said ALI system at a second instant in time 
following said first instant in time;, and 

comparing . said second reported location with said second 
ground truth to determine said accuracy of said ALI system to 
track movement of said wireless communication device. . 

13. A method for evaluating accuracy .of an automatic 
•location identification (ALI) system configured to locate a 
wireless communication .device originating an emergency call 
through a wireless communication network, said method 
comprising: 

identifying a validation region in which a service area of 
a public safety answering point (PSAP) and a radio frequency 
(RF) coverage area of said wireless communication network 
overlap; • 

selecting a test scenario that represents a calling 
environment in which said emergency call may occur; 
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choosing a test point in said validation region exhibiting 
said. calling environment represented by said test scenario ; 

determining a ground truth of said test point; 

performing a test call from .said test point, said test call 
simulating said emergency call; 

obtaining a reported location of said test point from said 
ALI system in response to said performing operation; and 

comparing said reported-location with said ground truth' to 
-determine said accuracy of said ALI system in -response to 
predetermined accuracy parameters. ; - 

14 . A method as claimed in claim 13 wherein said 
identifying operation comprises:; ' •': 

/ obtaining a first map of said service area; 

acquiring a second map of said RE coverage area; 

selecting unit areas that are common to each of said first 
and second maps; and / y 

compiling said unit areas to form a validation region map 
identifying said validation region . 

15. A method as claimed in claim 14 wherein said acquiring 
.operation comprises: i 

defining a wireless, service area of said wireless 
communication network approximately corresponding to said ' 
service area of said PSAP; 

locating RF coverage holes within said wireless service 
area, said RF coverage holes being regions of said wireless 
service area exhibiting a signal strength of RF communication 
signals that is less than a predetermined design threshold; and. 
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dissociating said RF coverage holes from said wireless 
service area to produce; said second map of said RF coverage ... 
area . 

16. A method as claimed in claim 15 wherein said 
identifying operation further comprises: : 

accessing a PSAP service area -database to obtain said first • 
map; . 

accessing a service area database of said wireless • - : 
communication network to define said wireless service area; 

. accessing an RF coverage area database that identifies said 
RF coverage holes within said wireless service area; and ••-V""-' V 

querying each of said PSAP service area database, said ■ 1 f 

service area database, and said RF coverage area database to 
compile said unit areas that arev common to each of said ; first •> 
and second maps, said unit area's exhibiting RF signal strength 
of said RF communication signals that is greater than said «, 
predetermined design threshold. \ 

17. A method as claimed in claim , 13 : further comprising: f 
obtaining a validation region* map of said validation 

region; . - - , . 

populating said validation region map with geographic and 
land use features of said validation region; 

partitioning said validation region map into a plurality of 
sub-regions; and J 

correlating each of said sub-regions with one of a 
plurality of test scenarios, said selected test scenario being 
one of said plurality of test scenarios, and each of said test 
scenarios representing a unique calling environment in said 
validation region. 
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18. A method as claimed in claim 17 wherein said choosing 
operation comprises initiating a random selection process to 
choose said test point from within said sub-regions correlated 
with said selected test scenario. 

19. A method as claimed in claim 17 wherein said choosing 
operation comprises distinguishing said test point in response 
; to a frequency of wireless emergency calls previously 
originated within a boundary of one of said sub-regions v 
•correlated with said selected test scenario relative to. said 
frequency of said wireless emergency calls previously - 
originated within others of said sub-regions correlated with 
said selected test scenario . » 

20. A method as claimed in claim 17 wherein said choosing 
operation comprises distinguishing said test point in response 
to a frequency of wireless calls previously conducted within a 
boundary of one of said sub-regions correlated with said 
selected test .scenario relative to said frequency of said 
wireless, calls previously conducted within others of said sub- 
regions correlated with said selected test scenario. 

21. A method as . claimed in claim 13 wherein said choosing 
operation comprises ascertaining that said test point is a 
viable test point for performing said test call, said viable 
test point being accessible and exhibiting said calling 
environment represented by said test scenario. 
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22. A method as claimed in claim 21 further comprising 
conducting, a ground survey of said test point to ascertain that 
said test point is said viable test point. 

23. A method as claimed in claim 13 wherein: v 
said method further comprises: 

i obtaining a validationLregion map of- said validation 
region; ? . .;: 

establishing an RF signal propagation model of said 
wireless communication network in said validation 
<'• .. region map,.. said,.mo,del representing propagation of RF 

communication signals through; said. validation region; 
activating said ALl system within said simulated 

environment ; and 
.locating said test point in said validation region map, 
said ground truth being determined from said validation 
region map; and 
said performing operation comprises simulating said test 
call within said simulated environment to obtain said reported 
location of said test point:. . 

' . • 24 ; A method as claimed in claim ;13. -wherein said test point 
is a first test point, said determining operation determines 
.said ground truth of said first test point at a first instant 
in time, said obtaining operation obtains said reported 
location of said first test point at said first instant in 
time, and said method further comprises: 

continuing said test call at a second test point; 

determining a second ground truth of said second test 
point; 
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obtaining a second reported location of said second test 
point from said ALI system at a second instant in time 
following said first instant in time; and 

comparing said second reported location with said second 
ground truth to .determine said accuracy of said ALI system to 
track movement of said wireless communication device. .1 

25. A computer-based method for evaluating accuracy of an 
automatic .location identification (ALT) system configured to. 
locate a wireless communication device. originating a call 
.through a? wireless c qmmun i c a t i on me two rk , said computer-based. 
method comprising: j : ;• /' 

identifying a validation region in which a service area ;of. 
a public safety answering point (PSAP) and a radio frequency 
(RF) coverage area of said wireless communication network 
.overlap, said identifying operation comprising: 

obtaining a first map of said service area; ■■ 
acquiring a seicond map of said RF coverage area ; 
■/■*.-■ : selecting, unit areas, that are common to each of said - 
first and second maps; and 
compiling said unit areas to form 1 a validation region map 
; identifying, said validation region;/ 
populating said, validation region map with geographic and 
land use features of said validation region; 

partitioning said validation region map into a plurality of 
sub- regions ; 

correlating each of said sub-regions with one of a 
plurality of test scenarios;: 

selecting pne of said plurality of test scenarios that 
represents a unique calling environment in which said call may 
occur ; 
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choosing a test point in said validation region exhibiting 
said unique environment represented by said one test scenario; 

determining a ground truth of said test point; 
..performing a test call from said test point,, said test call 
simulating said call; 

obtaining a reported location of said test point from said 
ALI system in response to said performing ^operation; and 

comparing said reported location with said ground truth to 
determine said accuracy of said ALI system in response to 
predetermined accuracy: parameters . . •* . :V 

26 . A; computer-based method as claimed in claim. 25 wherein 
said acquiring operation comprises: 

defining a wireless service area of said wireless 
communication network approximately corresponding to said 
service area of said PSAP; \u 

locating RF coverage holes within said wireless service ; 
area, said RF coverage holes being regions of said wireless 
service area that exhibit a. signal strength of RF. communication 
signals that is less than a predetermined design threshold; and 

dissociating said RF coverage holes from said wireless 
service area- to produce said: second ,map. of . said' RF. coverage j 
area. 

27., A computer-based method as claimed in claim 25 wherein: 

said method further comprises: 

establishing an RF signal propagation model of said 
wireless communication network in said validation 
region map, said model representing propagation of RF 
cpmmunication signals, through said validation region; 
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activating said ALI system within said simulated 

environment; and 
locating said test point in said validation region map, 
" ?; ■ ■ said r ground truth being .determined from said validation 

region map;andi 
said performing operation comprises simulating said test ; 
call "within said simulated .environment to obtain said reported 
location. • v ; : . . 

28. A computer-based method as claimed in claim 25 further 
comprising 'executing. said , choosing, .determining, performing, 
obtaining,., and comparing operations for a plurality of test iV" 
points exhibiting said unique environment represented by said 
one test scenario class. ' . •• . 

29. A computer-based method as claimed in claim 25 further 
comprising: 

selecting a second one of said plurality of test scenarios 
that .represents a second unique calling environment in which < 
said call may occur; ■ ' ' 

choosing a, second test point in said validation region 
exhibiting saidv, second environment represented ;by said -second 
test scenario; and 

executing said determining, performing, obtaining, and 
comparing operations for said, second test point. 

. • ■ ■ ■ ■."*'". ' f 

30. A computer-based method for evaluating accuracy ,of an . 
automatic location identification (ALI) system configured to 
locate a : wireless communication device originating an emergency 
call through a wireless communication network, said computer- 
based method comprising: . 
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identifying a validation. region in which a service area of 
a public safety answering point (PSAP) and a radio frequency , 
(RF) coverage area of said wireless communication network 
overlap, said identifying operation, comprising: 

obtaining a first map of said service area; 
acquiring a second map of said RF coverage area; 
^selecting unit areas that are common to each of said 
first and second maps; and 
compiling said unit' areas to form a- validation region map 
identifying said validation region; 
. populating said validation region map ^ with geographic and 
land use features of said validation regions- 
partitioning said validation region map into a plurality of 
sub-regions; 

correlating each of said sub-regions with one of a 
plurality , of test /scenarios; ./ 

selecting one of said plurality of test scenarios that 
represents a unique calling environment in which said emergency 
call may occur; 

choosing a test point in said validation region exhibiting 
said unique environment represented by said one test scenario; 
determining a ground truth of said test point; 
performing a test call from said test point, said test call 
simulating said emergency call; 

obtaining a reported location of said test point from said 
. ALI system in response to said performing operation; and 

comparing said reported location with said ground truth to 
determine said accuracy of said ALI syistem in response to 
predetermined accuracy parameters . 
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31. A computer-based method as claimed in claim 30 wherein 
said acquiring operation comprises: 

defining a wireless service area of said wireless 
communication network approximately corresponding to said 
service, area of said PSAP; : 

locating RF coverage holes within said wireless service 
area, said RF coverage holes >being regions of - said wireless 
service area that exhibit a signal strength of RF communication 
signals 'that is less than a predetermined' design threshold; and 

dissociating said RF coverage holes from said wireless 
service area, to produce ; said, second map .of . .said RF .coverage ./. 
-area . : . ■ ^ -V _ y \ • • . 

32. A computer-based method as claimed in claim 30 wherein: 
said method further, comprises : 

■establishing an RF signal propagation model of said 
wireless communication network in said validation 
region map, said model representing propagation of RF 
communication signals through said validation region; 
/activating said ALI system within said simulated ' 
environment; and 
locating, said test point in 'said/validation region map, 
. said ground truth being determined from said validation 
region map; and 
said performing operation comprises simulating said test 
call within said simulated environment to obtain said reported 
location. 

33. A computer-based method as claimed in claim 30 further 
comprising executing said choosing, determining, performing, 
obtaining, and comparing operations for a plurality of test 
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points exhibiting said unique environment represented by said 
one test scenario class. 

34. A computer-based method as claimed in claim 30 further 
comprising: 

selecting a second one of said plurality of test scenarios 
that represents a second unique calling environment in which 
said emergency call may occur ; 

choosing a second test- point in said validation region 
exhibiting said -second environment represented by said second 
test Scenario;, and . ! . ...6 , . < t; 

executing said determining, performing, obtaining, and 
comparing operations for said second test point. 

35. A computing system for selecting test points in a 
validation region, said test points being used to, validate 
accuracy of an automatic location identification (ALI) system 
operating in said validation: region, said ALI system being 
configured to locate a wireless communication device 
originating a call through a wireless communication network, ■< , 
said computing system comprising: . ,/ 

a processor; '" ■ 

a computer-readable storage medium; and 

executable code recorded on said computer-readable storage 
medium for instructing said processor to select said test 
points, said executable code, including: 

a validation region map subprocess for forming a validation 
region map of said validation region, said validation region 1 
map representing an overlap of a service area of a public 
safety answering point (PSAP) and a radio frequency (RF) 
coverage area of said wireless communication network; 
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a sub-region classification subprocess for partitioning 
said validation region map into a plurality of sub-regions and 
correlating each of said sub-regions with one of a plurality lot 
test scenarios, each of said test scenarios representing a 
unique calling environment in which said .call may occur;;- and 

a test point selection .subprocess for choosing said test 
points in response to a selected one of said test- scenarios, 
said test points exhibiting said -unique calling environment 
represented by : said selected one of .said test, scenarios , ! i 

.36. A computing system as claimed in, claim. 35 wherein said 
validation region map subprocess contains instructions for : >• 

accessing a PSAP service area database to obtain said 
service area of said PSAP; r 

accessing a service, area database of said wireless 
communication network to define a wireless service area of said 
/network approximately corresponding to said service area of : 
said PSAP; :! . .' >, 

accessing an RF coverage area database to identify: said RF 

coverage area within said .wireless service area, said RF 

• - -. * ■ ■ • . . ■ k ■ ' ' ■ i 

•coverage area being a portion of said wireless service area 

that exhibits RF signal strength of said RF communication 

signals that is greater than said predetermined design 

threshold;. 

querying each of said PSAP service area database, said 
wireless service area database, and said RF coverage area 
database to compile unit areas to form said validation region 
map., said unit areas being common to" each of said service area 
of said P^SAP, said wireless service area, and said RF coverage 
area. ' 
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37. A computing system as claimed in claim 35 wherein said 
sub-region classification subprocess contains instructions for 
populating said validation region map with geographic and land 
use features of said validation region to form said unique 
calling environments represented by said test scenarios, 

38. A computing system: as .claimed in claim. 35 wherein said 
test point selection subprocess contains instructions for 
initiating a" random ■■ selection process to choose- said test * i 
points from a plurality of points positioned within said sub- 
regions correlated with; said .selected - test scenarios. 

<v 39. A computing system as claimed in claim 35 wherein said 
call is an emergency call and said a test point selection 
subprocess contains instructions: for distinguishing said test 
points in response to a frequency of wireless emergency calls 
previously originated within a boundary of one of said sub- r 
regions correlated with said selected test scenario relative to 
said frequency of said wireless emergency calls previously 
originated within others of;, said, sub-regions correlated with . 
said selected test scenario . 

. 40. A computing system as claimed in claim- 35 wherein said 
call is an emergency call and said a test point selection 
subprocess contains instructions for distinguishing said test 
points in response to a frequency of wireless calls previously 
conducted within a boundary of one of said sub-regions 
correlated with said selected test scenario relative to said 
frequency of said wireless calls previously conducted within 
others of said sub-regions correlated with said selected test 
scenario. 
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. 41. A computing system for selecting test points in a 
validation- region, said test points being used to validate 
accuracy of an automatic location identification (ALI) system 
operating in said validation region, said ALI system being ■■. 
configured to locate a wireless communication device 
originating an emergency call through a wireless communication 
network, said computing system comprising: 

a processor; ; : m Ic . 
, a- computer-readable storage medium; and 

executable code recorded on said computer-readable storage 
medium . for .instructing ^said processor to select said test 
points, said executable code, including: v 

a validation region^ map subprocess for forming a validation 
region map of said validation region, . said validation region - 
map representing an overlap of a service area of a public 
safety answering point (PSAP) and a radio frequency (RF) . Z 
coverage area of said wireless communication network; 

a. sub-region classification subprocess for partitioning 
said validation region map into a plurality of sub-regions and 
correlating, each of ; said sub-regions ;with one of a plurality of 
test scenarios, each of said test, scenarios representing a 
-unique calling environment in which said emergency call may 
! occur; and 

♦a;-, test point selection subprocess for choosing said test- 
points in response to a selected one of said test scenarios, 
said test points exhibiting said unique calling environment 
represented by said selected one of said test scenarios. 
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42. A ^computing; system as claimed in claim 41 wherein said 
validation region map subprocess contains instructions for: ■ 

accessing a PSAP service area database to obtain said 
service area of said PSAP;' 

accessing a service area database\of said wireless 
communication network to define a wireless service area of said 
network approximately corresponding tov said service area of 
said PSAP; . ; 

accessing an RF coverage area database to identify said RF 
coverage area within said wireless service area, said RE 
coverage area being; a portion of ; said wireless service area •'. 
that exhibits RF signal strength of said RF communication 
signals that is greater .than said predetermined design 
threshold; . • : l , ■;}. 

querying each of said PSAP service area database, said 
wireless service area database, and said RF coverage area 
database to compile unit areas to form said validation region 
map, said unit areas being, common to each of said service area 
of said PSAP, said wireless service area, and said RF coverage 
area.- ■ '■' 

43 . A computing system, as claimed in claim 41 wherein said 
sub-region classification subprocess contains instructions for 
populating said validation region map with geographic and land 
use features of said validation region to form said unique . 
calling environments represented by said test scenarios. 

44. A computing system as claimed in claim 41 wherein said 
test point selection subprocess contains instructions for 
initiating a random selection process to choose said test 
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points from a plurality of points positioned within said sub- 
regions correlated with said selected test scenarios. 

.45,. A computingrsystem as claimed in claim 41 wherein said 
a ? test point selection subprocess contains instructions for 
distinguishing: said test points in response to a frequency of 
wireless emergency calls previously, .originated within a 
boundary of one of said sub-regions correlated with said 
selected test scenario 'relative to said frequency of said 
wireless emergency calls previously . originated within others, of 
said sub-regions .correlated with said selected test scenario . 

46. A computing system as claimed in claim 41 wherein said 
a test point selection subprocess contains instructions for 
distinguishing said test points in response to a frequency of 
wireless calls; previously conducted within a boundary of one of 
said , sub-regions correlated with said selected test scenario 
relative to said frequency of said wireless calls previously 
conducted within others of said sub-regions • correlated with 
said selected test scenario. 



